One of the recent trends emerging in the Data Warehousing marketplace is further integration of Data Warehouses into the Enterprise Information Technology (IT) infrastructure. Active Data Warehousing is finding wider acceptance across different industries. The origin of the active data warehouse is the timely, integrated store of detailed data available for tactical and strategic business decision making. The “active” traits are supplemental to traditional data warehouse functionality. For example, the complex, strategic decision queries normally found in traditional data warehouse can be expanded to support short, tactical queries, with real-time data feeds, and possibly event-driven updates, all occurring at the same time.
One of the major differences between an active data warehouse and a traditional data warehouse is that the former lacks the clearly identified load and query cycles; which mean the data can be continuously loaded and continuously queried, usually with 24 hours by 7 days per week (24×7) availability. Analytical applications, too, may be running continuously. Source data may appear at unpredictable times. System resources and requirements can change while jobs are running. Long running jobs may need to be monitored for their ad-hoc data access and acquisition, suspended for system resource constraints, and resumed when those resources become available. In addition, high data freshness and data consistency are also important requirements for the success of the Active Data Warehouse. As such, data warehousing components are required to be more intelligent and controllable, in real-time or in batch, in terms of what data is to be optimally accessed (or processed) at what time. Job status and events need to be monitored, and sometimes need to be acted on at the right time to ensure efficient and timely data access and loading.
To address these issues, a flexible software component is needed, which would allow users to monitor, modify, and control the performance and resource parameters of running jobs such that job events can be handled and system resources can be used in an effective manner. This is all due to the fact that there is a much higher demand for real-time tactical decision queries in the active data warehousing environment than for strategic decision queries in the traditional data warehouse environment.
Thus, improved mechanisms for asynchronous command processing in a parallel processing environment are desirable.